A Fast Spectral Relaxation Approach to Matrix Completion via Kronecker Products

In the existing methods for solving matrix completion, such as singular value thresholding (SVT), soft-impute and fixed point continuation (FPCA) algorithms, it is typically required to repeatedly implement singular value decompositions (SVD) of matrices. When the size of the matrix in question is large, the computational complexity of finding a solution is costly. To reduce this expensive computational complexity, we apply Kronecker products to handle the matrix completion problem. In particular, we propose using Kronecker factorization, which approximates a matrix by the Kronecker product of several matrices of smaller sizes. We introduce Kronecker factorization into the soft-impute framework and devise an effective matrix completion algorithm. Especially when the factorized matrices have about the same sizes, the computational complexity of our algorithm is improved substantially. Introduction The matrix completion problem (Cai, Candès, and Shen 2010; Candès and Recht 2008; Keshavan, Montanari, and Oh 2009; Mazumder, Hastie, and Tibshirani 2010; Beck and Teboulle 2009) has become increasingly popular, because it occurs in many applications such as collaborative filtering, image inpainting, predicting missing data in sensor networks, etc. The problem is to complete a data matrix from a few observed entries. In a recommender system, for example, customers mark ratings on goods and vendors then collect the customer’s preferences to form a customer-good matrix in which the known entries represent actual ratings. In order to make efficient recommendations, the vendors try to recover the missing entries to predict whether a certain customer would like a certain good. A typical assumption in the matrix completion problem is that the data matrix in question is low rank or approximately low rank (Candès and Recht 2008). This assumption is reasonable in many instances such as recommender systems. On one hand, only a few factors usually contribute to an customer’s taste. On the other hand, the low rank structure suggests that customers can be viewed as a small number of groups and that the customers within each group have similar taste. Copyright c © 2011, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Recently, it has been shown that matrix completion is not as ill-posed as originally thought. Srebro, Alon, and Jaakkola (2005) derived useful generalization error bounds for predicting missing entries. Several authors have shown that under certain assumptions on the proportion of the missing entries and locations, most low-rank matrices can be recovered exactly (Candès and Recht 2008; Candès and Tao 2009; Keshavan, Montanari, and Oh 2009). The key idea of recovering a low-rank matrix is to solve a so-called matrix rank minimization problem. However, this problem is NP-hard. An efficient approach for solving the problem is to relax the matrix rank into the matrix nuclear norm. This relaxation technique yields a convex reconstruction minimization problem, which is tractably solved. In particular, Cai, Candès, and Shen (2010) devised a first-order singular value thresholding (SVT) algorithm for this minimization problem. Mazumder, Hastie, and Tibshirani (2010) then considered a more general convex optimization problem for reconstruction and developed a softimpute algorithm for solving their problem. Other solutions to the convex relation problem include fixed point continuation (FPCA) and Bregman iterative methods (Ma, Goldfarb, and Chen 2009), the augmented Lagrange multiplier method (Lin et al. 2010; Candès et al. 2009), singular value projection (Jain, Meka, and Dhillon 2010), accelerated proximal gradient algorithm (Toh and Yun 2009), etc.. These methods require repeatedly computing singular value decompositions (SVD). When the size of the matrix in question is large, however, the computational burden is prohibitive. Implementations typically employ a numerical iterative approach to computing SVD such as the Lanczos method, but this does not solve the scaling problem. In this paper we propose a fast convex relaxation for the matrix completion problem. In particular, we use a matrix approximation factorization via Kronecker products (Van Loan and Pitslanis 1993; Kolda and Bader 2009). Under the nuclear norm relaxation framework, we formulate a set of convex optimization subproblems, each of which is defined on a smaller-size matrix. Thus, the cost of computing SVDs can be mitigated. This leads us to an effective algorithm for handling the matrix completion problem. Compared with the algorithms which use numerical methods computing SVD, our algorithm is readily parallelized. The paper is organized as follows. The next section reProceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence